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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 14 October 2004 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 
closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) D Claim(s) 1-15 and 20-27 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) IEI Claim(s) 1-15,20.22,24 and 26 is/are rejected. 

7) £3 Claim(s) 21.23,25 and 27 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)£3 The drawing(s) filed on 16 April 2001 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.0 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Response to Arguments 

1. Applicant's arguments with respect to claims 1-15 and 20-27 filed on 10/14/04 have been 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-3, 5, 7-9, 1 1, 13, 20, 22, 24 and 26 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Brais et al. U.S. Patent 5,995,936 in view of Bregler U.S. Patent 
5,880,788. 

Referring to claim 1, the Brais reference discloses in Figures 1 and 9, a digital camera 
system (See CoL 5, lines 5-10), comprising: a housing (camera 104, see Col. 6, lines 16-18); 
a digital optical sensing apparatus mounted within said housing, said digital optical sensing 
apparatus sensing optical images; a storage medium (computer 102) for storing digital optical 
images captured by said digital optical sensing apparatus (See Col. 6, lines 65-67); an 
acoustic sensor (transducer 106, See CoL 10, lines 40-41) capable of sensing human speech; 
a speech reduction apparatus (inside of computer 102) coupled to said acoustic sensor, said 
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speech reduction apparatus converting human speech sensed by said acoustic sensor to a 
symbolic text form(See Col. 10, lines 40-46); and a controller which stores said symbolic text 
form in said storage medium in a relationship associated with a captured digital image (image 
identifier in the form of filename is added to the text file) as shown in Figure 9 (See Col. 11, 
lines 39-55). However, the Brais reference does not explicitly show each of multiple images 
is associated with a discrete segment of a plurality of discrete segments of text. 

The Bregler reference teaches in Figures 3 and 8, an image system receives a user 
indication of a plurality of discrete time intervals (e.g., audio track 14 is a time base signal 
including a phonetic words as a plurality of discrete segments), records a plurality of discrete 
human speech segments sensed by a acoustic sensor in respective said discrete time intervals 
(e.g., time warping as shown in Figure 8, see Col. 10, lines 15-30); causes said speech 
reduction apparatus to convert each said human speech segment to a corresponding symbolic 
text segment (by speech recognizer 16, See Col. 5, lines 37-41, and Col. 6, lines 1-35); and 
automatically associates a respective digital optical image with each said symbolic text 
segment (phonetic string or full pronunciations of the phone) based on a temporal 
relationship between the time interval in which the discrete human speech segment 
corresponding to the symbolic text segment was recorded (e.g., aligning each of image 
segments with a corresponding symbolic text segment must involve time controlling, and 
time controlling involves with adjusting time interval in order to realize synchronization, See 
Col. 5, lines 40-61). The Bregler reference is evidence that one of ordinary skill in the art at 
the time to see more advantages the system can control each of multiple images associated 
with a discrete segment of a plurality of discrete segments of text based on a temporal 
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relationship between the time interval in which the discrete human speech segment so that 
providing a technique which permits any given sound utterance to be substituted for the 
soundtrack of a previously recorded video sequence, without requiring a video recording of 
the new sounds being uttered; and it is also further desirable to provide such a method which 
readily lens itself to automation, to thereby minimize the amount of manual input that is 
required. For that reason, it would have been obvious one of ordinary skill in the art at the 
time to modify the digital camera system of Brais for providing a controller to control each 
of multiple images associated with a discrete segment of a plurality of discrete segments of 
text based on a temporal relationship between the time interval in which the discrete human 
speech segment as taught by Bregler. 

Referring to claim 2, the Brais and Bregler references disclose all subject matter as 
discussed in respected to claim 1, and the Brais reference discloses wherein said controller 
(inside of computer 102) comprises a programmable processor executing a control program 
(acquire image mode program) for controlling the operation of said digital camera (a 
command to acquire an image,. See Col. 11, lines 51-54). 

Referring to claim 3, the Brais and Bregler references disclose all subject matter as 
discussed in respected to claim 1, and the Brais reference discloses wherein said speech 
reduction apparatus comprises a speech reduction algorithm embodied as a plurality of 
instructions executable on said programmable processor (See Col. 10, lines 41-46). 

Referring to claim 5, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 1. 
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Referring to claim 7, the Brais and Bregler references disclose all subject matter as 
discussed in respected to claim 5, and the Brais reference discloses wherein said step of 
rendering said human speech in a symbolic text form is performed by a programmable 
processor executing a speech reduction program inside of computer 102 (See Col. 9, lines 31- 
35 and Col 10, lines 21-28). 

Referring to claim 8, the Brais and Bregler references disclose all subject matter as 
discussed in respected to claim 7, the Brais reference discloses wherein said programmable 
processor further executes a control program for controlling the operation of said digital 
camera, and said step of rendering said human speech in a symbolic text form is performed 
by said programmable processor in the background (While the text is an image identifier and 
inserted in image) when said control program is otherwise unoccupied (See Col. 13, lines 27- 
35) and the Bregler references discloses the segment of said plurality of discrete segments of 
human speech in respective corresponding segment of symbolic text as shown in Figure 3. 

Referring to claim 9, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 1. 

Referring to claim 11, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 1, and the Brais reference discloses 
uploading said at least one digital image and said at least one segment of symbolic text to a 
digital image formatting apparatus; and formatting said at least one digital image and said at 
least one segment of symbolic text for viewing by a user using said digital image formatting 
apparatus (any appropriate software application, i.e., Microsoft PowerPoint to format to 
world wide web pages or any multimedia files), wherein each said segment of symbolic text 
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is formatted for viewing in a human readable form (report) associated with its corresponding 
digital image (See Col. 12, lines 9-23). 

Referring to claim 13, the Brais reference discloses wherein said digital image formatting 
apparatus is a general-purpose digital computer (102) executing a digital image- formatting 
program (i.e., Microsoft PowerPoint, See Col. 12, lines 12). 

Referring to claim 20, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 1 , and the Bregler reference discloses 
wherein said controller associates a respective digital image with each symbolic text segment 
according to the following association: if a first digital image is captured during the recording 
of a human speech segment corresponding to the symbolic text segment, the symbolic text 
segment is associated with the first digital image as shown in Figures 3 and 8 (e.g., the 
Bregler reference teaches the system can automatically associates the symbolic text segment 
with the captured image responding to a human speech segment, such as speaker's lip 
position that corresponds to each spoken sound, see Col. 5, lines 55-61). 

Referring to claim 22, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 20. 

Referring to claim 24, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 20. 

Referring to claim 26, the Brais and Bregler references disclose all subject matter as 
discussed in respected to same comments with claim 20. 
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4. Claims 4, 6, 10 and 12 rejected under 35 U.S.C. 103(a) as being unpatentable over Brais et 
aL U.S. Patent 5,995,936 in view of Bregler U.S. Patent 5,880,788 and Williams U.S. Patent 
6,308,154. 

Referring to claim 4, the Brais and Bregler references disclose all subject matter as 
discussed in respected claim 1, except that the Brais reference does not explicitly provide any 
detail of the speech to text conversion such as converts the human speech to an intermediate 
symbolic form comprising a symbolic representation of phonemes, the intermediate symbolic 
form being subsequently reduced to natural (spoken) language text by a separate apparatus. 

The Williams reference discloses in Figures 1-3, a method for encoding a human speech 
under a symbolic textual format (e.g., phonemes, morphemes, words, sentences etc., See Col. 
2, lines 1-10), converting into a natural (spoken) language text by a separate apparatus (CPU 
18 including speech recognition application 24 identify individual words and recognizing 
phonetic elements. When words are recognized, the CPU 18 store the individual words as 
textual information, and textual information subsequently reduced to natural language text 
with speech attributes, See Col. 2, lines 1-16, lines 40-53 and Col. 3, lines 3-7). The 
Williams reference is evidence that one of ordinary skill in the art at the time to see more 
advantages for a speech recognition apparatus can be programmed to phonetically break 
down the words so that a reader would have enough information to discern the meaning 
conveyed and understanding which meaning was intended. For that reason, it would have 
been obvious one of ordinary skill in the art to see the speech reduction apparatus (102) 
converts said human speech sensed by said acoustic sensor to an intermediate symbolic form 
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comprising a symbolic representation of phonemes, said intermediate symbolic form being 
subsequently reduced to natural language text by a separate apparatus disclosed by Brais. 

Referring to claim 6, the Brais and Williams references disclose all subject matter as 
discussed in respected claim 4. 

Referring to claim 10, the Brais and Williams references disclose all subject matter as 
discussed in respected claim 4. 

Referring to claim 12, the Brais and Williams references disclose all subject matter as 
discussed in respected claim 4. 

5. Claims 14-15 rejected under 35 U.S.C. 103(a) as being unpatentable over Brais et al. U.S. 
Patent 5,995,936 in view of Bregler U.S. Patent 5,880,788 and Englehardt U.S. Patent 
5,477,511. 

Referring to claims 14-15, the Brais and Bregler references disclose all subject matter as 
discussed in respected claim 11, and the Brais reference shows the digital camera system can 
format the digital image and the symbolic text (converted from human speech) to a 
multimedia report or database file (See Col. 11, lines 45-48). However, the Brais reference 
does not explicitly show a printer for printing out the report file on paper or a display screen 
for viewing the report file. 

The Englehardt reference discloses in Figures 1 and 4, a digital camera system 
comprising: recording visual information (image) by CCD (16); receiving voice data and 
transcribing the voice data by computer (5) voice recognition software; the both image data 
and text data format in a documentation; a printer (52) coupled to the system for printing out 
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the documentation; and the computer display screen to viewing the documentation.(See Col. 
5, lines 29-44). The Englehardt reference is evidence that one of ordinary skill in the art at 
the time to see more advantages for digital camera system has a printer and display screen for 
user to obtain a hard copy or quick preview immediately after the multimedia report is 
completed. For that reason, it would have been obvious to see the digital camera has a printer 
for printing out the report file on paper or a display screen for viewing the report file 
disclosed by Brais. 



Allowable Subject Matter 

6. Claims 21, 23, 25 and 27 would be allowable if rewritten to overcome the rejection(s) under 
35 U.S.C. 112, 2nd paragraph, set forth in this Office action and to include all of the 
limitations of the base claim and any intervening claims. 

7. The following is an examiner's statement of reasons for allowance: 

The prior art does not teach or fairly suggest the controller associates a respective digital 
image with each symbolic text segment according to all of the following association 
priorities: (1) if a first digital image is captured during the recording of a human speech 
segment corresponding to the symbolic text segment, the symbolic text segment is associated 
with the first digital image; (2) if no digital image is captured from a time the digital camera 
is powered on until the end of the recoding of the human speech segment corresponding to 
the symbolic text segment, and a second digital image is captured after recording the human 
speech segment but before the digital camera is powered off, then the symbolic text segment 
is associated with the second digital image; and (3) in all other cases, the symbolic text is 
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associated with the digital image most recently captured before the recording of the human 
speech segment corresponding to the symbolic text segment. 

Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until 
after the end of the THREE-MONTH shortened statutory period, then the shortened statutory 
period will expire on the date the advisory action is mailed, and any extension fee pursuant to 
37 CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of 
this final action. 

9. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Lin Ye whose telephone number is (571) 272-7372. The examiner can 
normally be reached on Mon-Fri 8:00AM-5:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, James J. Groody can be reached on (571) 272-7950. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access 
to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 
(toll-free). 
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PRIMARY EXAMINER 
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April 8, 2005 



